Interactive computing advice facility through third-party data

ABSTRACT

In embodiments of the present invention improved capabilities are described for helping a user make a decision through the use of a computing facility, where the computing facility may be a machine learning facility. The process may begin with an initial question being received by the computing facility from the user. The user may then be provided with a dialog consisting of questions from the computing facility and answers provided by the user. The computing facility may then provide a decision to the user based on the dialog and pertaining to the initial question, such as a recommendation, a diagnosis, a conclusion, advice, and the like. In embodiments, future questions and decisions provided by the computing facility may be improved through feedback provided by the user. In embodiments, the present invention may be utilized in conjunction with a third-party application.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of the following provisionalapplication, which is hereby incorporated by reference in its entirety:U.S. Provisional App. No. U.S. 61/097,394 filed Sep. 16, 2008.

This application is a continuation of the following U.S. patentapplication: U.S. application Ser. No. 12/483,768 filed Jun. 12, 2009;which is a continuation-in-part of the following U.S. patentapplication: U.S. application Ser. No. 12/262,862 filed Oct. 31, 2008which claims the benefit of the following applications, U.S. ProvisionalApp. No. 60/984,948 filed Nov. 2, 2007 and U.S. Provisional App. No.61/060,226 filed Jun. 10, 2008. Each of these applications isincorporated herein by reference in its entirety.

BACKGROUND

1. Field

The present invention is related to collective knowledge systems, andmore specifically to providing natural language computer-based topicaladvice based on machine learning through user interaction.

2. Description of the Related Art

Online searching for topical advice represents a significant use ofcomputer resources such as provided through the Internet. Computer usersmay currently employ a variety of search tools to search for advice onspecific topics, but to do so may require expertise in the use of searchengines, and may produce voluminous search results that take time tosift through, interpret, and compare. People may be accustomed to askingother people for advice in spoken natural language, and therefore it maybe useful to have a computer-based advice tool that mimics more closelyhow people interact with each other. In addition, advice on topics maychange in time, and any static database of advice may fall quickly outof date. Therefore, a need exists for improved topical advice searchcapabilities adapted for use with natural language, and that providesfor continuous content refinement.

SUMMARY

The present invention may consist of a computing facility, such as amachine learning facility, that may ask a user questions, and based onthe user's answers the system may offer a decision, such as arecommendation, a diagnosis, a conclusion, advice, and the like.Internally, the system may use machine learning to optimize whichquestions to ask and what decision to make at the end of the questionand answer dialog. The system may learn through users providing feedbackon the provided decision, including deciding whether the decision washelpful or not. Helpful decisions may become reinforced and becomeassociated with the questions and answers that were asked along the way.When a user indicates that a decision was helpful, the system mayremember which questions it asked, what the answer to each question was,and may associate these questions and answers with the ultimatedecision. In embodiments, these associations may be the basis of themachine learning that may learn over time which question to ask the nexttime a user comes to the system.

In embodiments, the present invention may help a user make a decisionthrough the use of a machine learning facility. The process may beginwith an initial question being received by the machine learning facilityfrom the user. The initial question may be received via a searchinterface, where the ultimate decision is based on the initial searchterms, the dialog of questions and answers with the user, the trainingin the system, and the like. The user may then be provided with a dialogconsisting of questions from the machine learning facility and answersprovided by the user. The machine learning facility may then provide adecision to the user based on the dialog and pertaining to the initialquestion, such as a recommendation, a diagnosis, a conclusion, advice,and the like. In embodiments, future questions and decisions provided bythe machine learning facility may be improved through feedback providedby the user.

In embodiments, the initial question posed to the user may be anobjective question, a subjective question, and the like. A question maybe provided from amongst a broad category of topics, such as topicspertaining to a product, personal information, personal health, economichealth, business, politics, education, entertainment, the environment,and the like. The question may be in the form of a multiple choicequestion, a yes-no question, a rating, a choice of images, a personalquestion, and the like. The question may be about the user, provided byanother user, provided by an expert, and the like. The question may bebased on a previous answer, such as from the current dialog with theuser, from a stored previous dialog with the user, from a storedprevious dialog with another user. The question may be a pseudo randomquestion, such as a test question, an exploration question that helpsselect a pseudo random decision on the chance that the pseudo randomdecision turns out to be useful, and the like. The question may includeat least one image as part of the question. The question may be alongpsychographic dimensions. In embodiments, the question may not be askeddirectly to the user, but rather determined from contextual information,such as through an IP address, the location of the user, the weather atthe user's location, a domain name, related to path information, relatedto a recent download, related to a recent network access, related to arecent file access, and the like.

In embodiments, the dialog may continue until the machine learningfacility develops a high confidence in a reduced set of decisions, suchas a reduced set of decisions presented to the user, a single decisionpresented to the user. The decision provided by the machine learningfacility may be independent of the order of questions in the dialog. Thedecision may provide an alternate decision when at least one question inthe dialog is omitted, where the alternate decision may be differentbased on the machine learning facility having less information from theuser. The decision may display a ranking of decision choices, such asranking decisions across non-traditional feature dimensions. Thedecision may display at least one image related to the decision. Thedecision may be a pseudo random decision on the chance that the pseudorandom decision turns out to be useful, such as the pseudo randomdecision being part of a system of exploration, where the system ofexploration may improve the effectiveness of the system, the machinelearning facility may learn from exploration, and the like.

In embodiments, the feedback provided may be related to, or derivedfrom, how the user answers questions in the dialog, how the userresponds to the decision provided by the machine learning facility, andthe like. In embodiments, the feedback may be solicited from the user.

In embodiments, users may extend the learning of the machine learningfacility by entering new information, where the new information may betheir own topic, question, answer, decision, and the like. The machinelearning facility may use the new information to determine whether thenew information is helpful to users.

In embodiments, expert users may extend the learning of thecomputational facility by entering new information, where the newinformation may be their own topic, question, answer, decision, and thelike. An expert user entering a new question, answer, or decision mayspecify how the system should rank the decisions when future users giveparticular answers to the question. The expert user may also specifypre-conditions/dependencies on when the new question should be asked.The expert user may also optionally enter an importance for thequestion.

In embodiments, the system may be implemented as a series of dynamicdecision trees. During the question and answer dialog with the user, thesystem may be looking at which questions are relevant to ask given theanswers the user has already provided. When users enter new questions,answers, or decisions they may specify which questions are relevant towhich decisions and how the answers affect the relative ranking of thevarious decisions.

In embodiments, the system may recommend a decision based on multipleindependent factors, such as how well a decision matches the objectiverequirements specified by the user during the question and answer dialogand how well the decision matches the user's subjective requirementssuch as the user's taste preferences.

In embodiments, a user interface may be provided for user interactionwith the machine learning facility, such as associated with a webinterface, instant messaging, a voice interface, a cell phone, with SMS,and the like.

In embodiments, the present invention may help a user make a decisionthrough the use of a machine learning facility. The process may beginwith an initial question being received by the machine learning facilityfrom the user, where the initial question may be associated with one ofa broad category of topics, such as product, personal, health, business,political, educational, entertainment, environment, and the like. Theuser may then be provided with a dialog consisting of questions from themachine learning facility and answers provided by the user. The machinelearning facility may then provide a decision to the user based on thedialog and pertaining to the initial question, such as a recommendation,a diagnosis, a conclusion, advice, and the like. In embodiments, futurequestions and decisions provided by the machine learning facility may beimproved through feedback provided by the user.

In embodiments, the present invention may help a user make a decisionthrough the use of a computing facility. The process may begin with aninitial question being received by the computing facility from the user.The user may then be provided with a dialog consisting of questions fromthe computing facility and answers provided by the user. The computingfacility may then provide a decision to the user based on an aggregatedfeedback from a plurality of users. In embodiments, the computerfacility may improve future questions and decisions provided by thecomputing facility based on receiving feedback from the user.

In embodiments, the present invention may help a user make a decisionthrough the use of a machine learning facility. The process may beginwith an initial question being received by the machine learning facilityfrom the user. The user may then be provided with a dialog consisting ofquestions from the machine learning facility and answers provided by theuser, where the number of questions and answers provided through thedialog may determine the quality of the decision. The machine learningfacility may then provide a decision to the user based on the dialog andpertaining to the initial question, such as a recommendation, adiagnosis, a conclusion, advice, and the like. In embodiments, futurequestions and decisions provided by the machine learning facility may beimproved through feedback provided by the user. In embodiments, thequality may be high when the number of questions and answers large, suchas greater than 10 questions, greater than 15 questions, greater than 10questions, and the like. In embodiments, the quality may be good qualitywhen the number of questions and answers is small, such as less than 10questions, less than 5 questions, less than 3 questions, one question,and the like.

In embodiments, the present invention may make a decision through theuse of a machine learning facility. The system may include a machinelearning facility that may receive an initial question from the user, adialog facility within the machine learning facility providing the userwith questions and accepting answers from the user, the machine learningfacility providing a decision to the user, and the like. In embodiments,the decision provided to the user may be based on the exchange betweenthe user and the machine learning facility, and pertain to the initialquestion. Further, the machine learning facility may receive feedbackfrom the user to improve future questions and decisions provided by themachine learning facility.

These and other systems, methods, objects, features, and advantages ofthe present invention will be apparent to those skilled in the art fromthe following detailed description of the preferred embodiment and thedrawings. All documents mentioned herein are hereby incorporated intheir entirety by reference.

BRIEF DESCRIPTION OF THE FIGURES

The invention and the following detailed description of certainembodiments thereof may be understood by reference to the followingfigures:

FIG. 1 depicts a list of topics in the system from which users may getdecisions.

FIG. 2 depicts an example question that the system may ask a user.

FIG. 3 depicts an example picture question that the system may ask auser.

FIG. 4 depicts an example of the type of information the system may showthe user when making a particular decision.

FIG. 5 depicts an example of top lists for cameras.

FIG. 6 depicts a second example of a top list for cameras.

FIG. 7 depicts an embodiment of a user home page.

FIGS. 8 and 8A depict an embodiment of a user's remembered answers.

FIG. 9 depicts choices that a user may contribute expertise.

FIG. 10 depicts an example of a user question.

FIGS. 11 and 11A depict an embodiment of an answer format.

FIG. 12 depicts an example list of all decisions in a topic.

FIG. 13 depicts an embodiment process flow for the present invention.

FIG. 14 depicts an embodiment process flow for the present invention.

FIG. 15 depicts an embodiment of a block diagram for the presentinvention.

FIG. 16 depicts an embodiment contributor/expert interface home page.

FIG. 17 depicts an embodiment objective question to a user looking forhelp in a decision.

FIG. 18 depicts an embodiment of a decision result showing a particularrecommended decision.

FIG. 19 depicts an embodiment interface for users to set associationsbetween attributes and decision results.

FIG. 20 depicts an embodiment illustrating how a user may edit adecision result.

FIG. 21 depicts an embodiment showing prior revisions to content andchanges between two prior revisions.

FIG. 22 depicts an embodiment showing a question being edited by a user.

FIG. 23 depicts an embodiment showing the revision history forattributes.

FIG. 24 depicts an embodiment of a workshop interface where newly addedareas of advice may be displayed.

FIG. 25 depicts an embodiment where the system is asking the user asubjective question in order to learn the preferences of the user.

FIG. 26 depicts an embodiment showing an activity feed of recentactivity by contributors.

While the invention has been described in connection with certainpreferred embodiments, other embodiments would be understood by one ofordinary skill in the art and are encompassed herein.

All documents referenced herein are hereby incorporated by reference.

DETAILED DESCRIPTION

The present invention may ask the user 1314 questions 1320 and based onthe user's answers the system may offer a decision, such as arecommendation, a diagnosis, a conclusion, advice, and the like.Internally, the system may use machine learning to optimize whichquestions 1320 to ask and what decision 1310 to make at the end of theprocess. The system may learn through users giving feedback on theultimate decision, whether the decision 1310 was helpful or not. Helpfulsolutions may get reinforced and associated with the questions 1320 andanswers 1322 that were asked along the way. When a user 1314 says that adecision 1310 was helpful the system may remember which questions 1320it asked, what the answer 1322 to each question 1320 was, and mayassociate these questions 1320 and answers 1322 with the ultimatedecision. These associations may be the basis of the machine learningthat learns over time which question 1320 to ask the next time a user1314 comes to the system.

For example a user 1314 may try to get advice picking a bar to visit.The system may ask the question “How old are you?” and get the answer“in my 30s”. Ultimately, the system may show the user 1314 the decision“Kelley's Irish Bar”. Assume the user 1314 says this decision washelpful. The system will increase the association between the question“How old are you?”, the answer “in my 30s” and the decision “Kelley'sIrish Bar”. The next time a user 1314 comes to the site looking foradvice on a bar, the system will be more likely to ask the user 1314 the“How old are you?” question 1320 since in the past this question 1320was useful in helping the user. If the user 1314 answers the question1320 in the same way as the prior user 1314 (saying “in my 30s”) thenthe system will increase its belief that the ultimate decision is“Kelley's Irish Pub”.

The system may build a profile of each user's tastes, aestheticpreferences, etc. and learn via feedback which decisions 1310 are likedby which types of people. Alternatively, an expert user may specifywhich kinds of decisions 1310 are liked by which kinds of people.Learning user's taste profiles may happen through a separate processfrom the dialog of questions 1320 and answers 1322 asked by the systemin a specific topic. For example, a user 1314 may separately tell thesystem about their taste choices through a different question and answerdialog designed specifically to understand the user's aestheticpreferences.

A user 1314 may not want to spend the time to teach the system about allof their taste preferences, and so instead the system may learn, or anexpert may specify, which of all the taste questions 1320 are the mostimportant taste questions to ask in the context of the user 1314 makingone specific decision 1310. Out of the universe of all questions thesystem may know about finding out about taste profiles, for instance thesystem may have learned there are three specific questions 1310 that arebest for when the user 1314 is trying to find a sedan under $25,000.Alternately, there may be a completely different set of three tastequestions to ask a user 1314 who is interested in a SUV over $45,000.

A user 1314 may also only tell the system about their taste preferencesand not about any objective questions. In this case the system mayprovide a ranking of all the decisions 1310 in an area of advice basedpurely on taste. So instead of the user 1314 saying they want a $200point-and-shoot camera, effectively what the user 1314 would be doing issaying they want a camera that other urban 35 year old men who prefercomputers to sports want. Users 1314 may indicate this preference byusing a search interface and choosing an area of advice that isexplicitly labeled “cameras for urban men in their 30s” instead of the“which camera should I buy” area of advice. Alternatively, users 1314may indicate their interest in making a decision 1310 about cameras andthen opt to not answer any of the questions in the Q&A dialog from thesystem and thus the system will only have subjective information aboutthe user 1314 to use in recommending cameras to the user 1314.Alternatively, users 1314 may answer questions 1320 in the dialog thatare both objective and subjective and the system may then recommend acamera based on the combined objective data about the camera andsubjective data about the camera.

Users may also enter new questions, answers, and ultimate decisions. Thesystem may then try out the new questions 1320 with future users to seeif the questions 1320 turn out to be useful in helping those users. Forexample, a user 1314 of the bar recommendation service may contributethe question “Do you want a loud place or a quiet intimate setting?”.The system may decide to ask this question 1320 in a future use of thebar recommendation service and through the process outlined aboveobserve a correlation between the answers of this question 1320 andrecommendations that users find useful. On the other hand, a user 1314may contribute a question 1320 that has no value in helping users. Forexample, a user 1314 could contribute the question “Do you have a Canoncamera?”. The system may try this question 1320 out on future users andfail to notice any correlation between the answers to this question 1320and bar recommendations that users find helpful. In this case, thequestion 1320 may get asked less since it's not predictive of whetherone recommendation or another recommendation is helpful.

The system may keep asking questions 1320 until it feels it has a highconfidence in a few possible decisions. The system may also stop soonerif it feels like it has already asked too many questions 1320 and risksannoying the user. The system may also ask at least a minimum number ofquestions 1320 to avoid the user 1314 feeling that the system couldn'tpossibly have asked enough to make an intelligent decision.

The system may have a mechanism to tolerate incorrect answers from theuser. Incorrect answers may result from the user 1314 not understandingthe question, not understanding the answer 1322 or not knowing theanswer 1322 to the question. If the bulk of the answers given by theuser 1314 support a particular decision, the system may make thatdecision 1310 even though not all the user's answers support thatdecision.

In embodiments, the present invention may provide for at least one ofquestions 1320 and answers 1322 between the system and the user,decisions to users, and machine learning utilized to improve decisions.The system may provide for an improved way to generate questions 1320and answers 1322, an improved way to provide decisions to users, animproved way to utilize machine learning to improve questions 1320 anddecisions provided by a system, and the like, where any of thesecapabilities may be separately, or in combination, used as a standalonesystem or incorporated into a third party system as an improvedcapability. In embodiments, each of these improved capabilities mayutilize some form of machine learning as described herein. For example,the system may provide for an improved way to execute a question 1320and answer 1322 session with a user 1314 by learning under whatcircumstances the user 1314 is looking for certain information. Forinstance, it may be learned by the system that the weather is acondition under which users have a differentiated preference dependingon the time of day and the weather conditions. When it's raining duringthe day, and a user 1314 searches for movies, the user 1314 may be morelikely to be looking for movie tickets and locations where the movie isplaying. When it's raining during the night, and the user 1314 searchesfor movies, the user 1314 may be more likely to be looking for adescription of the movie. In another example, the system may provide foran improved way to provide decisions to users, such as learning thatusers prefer certain formats during the daytime versus during theevening, providing choices verses a single decision 1310 based on age,prefer a greater number of questions 1320 prior to presentation of thedecision 1310 based on the user's geographic location, and the like. Inanother example, the system may provide for an improved way to learnwhat decision 1310 to choose for a user, such as utilizing greaterexpert information based on age and education, utilizing popular opinionmore when the topic is fashion and the user 1314 is young versusutilizing traditional practice more when the user 1314 is older, askingmore questions 1320 about the user's choices in friends when the topicis personal, and the like.

In embodiments, the present invention may provide for combinations ofquestion 1320 and answer, providing decisions, and learning whatdecisions to provide, where one of the elements may not be provided bythe system, such as when that element is provided by a third partysystem. For example, a third party search engine web application maywhat to improve their capabilities for providing sorted lists from auser's search query, and so may want to utilize the present invention'sfacility for generating questions 1320 and answers 1322 to augment theirkeyword search and sort algorithms. In this instance, the third partysearch engine provider may not be interested in the present invention'sfacility for generating decisions, because their service is in thebusiness of providing sorted lists, not a limited set of decisions.However, the present invention may provide an important new capabilityto the search engine provider, in that the present invention's abilityto constantly improve the questions 1320 and answers 1322 to users mayenable the search engine provider to improve their sorting result tousers based on the present invention's capabilities.

In embodiments, the subject of the initial area of advice may bespecified through a search interface. For example, a user 1314 searchingfor “romantic honeymoons in Italy” may get taken to a web page thathelps the user 1314 decide where to honeymoon in Italy instead of firstasking the user questions about where they want to vacation, what typeof vacation they were looking for etc. Or a user 1314 could search for aspecific location in Italy and be directed to a web page on that 1)helps the user 1314 decide if that specific location is a good one fortheir needs (for example, showing things like “this vacation is good forhoneymooners and romantic getaways and bad for family vacations”) and 2)offers to start a dialog to help the user 1314 find alternative andpotentially better locations in Italy to vacation. Or a user 1314 couldbe searching for specific products and then enter into a dialog tonarrow down which of those products are best for them. In both cases #1and #2 the information shown may be based how other users have answeredquestions in decision making dialogs and then given positive feedback tothis decision. So if many people using the “where should I go onvacation” topic answered a question “do you want a romantic vacation”with “yes” and then gave positive feedback to “Italy” the system woulddisplay that Italy is a romantic destination to users 1314 coming in viasearch engines. Alternatively, the users 1314 who added the decision“Italy” or the question “do you want a romantic vacation” into thesystem could have explicitly indicated that the answer “yes” to thequestion “do you want a romantic vacation” should be associated withItaly and thus show that Italy is a romantic vacation to users 1314coming in via search engines.

In embodiments, the present invention may provide other combinations ofsome subset of asking questions, making decisions, and learning to makebetter decisions, such as using the present invention's facilities formaking better decisions, but only using input from experts; notproviding a question 1320 and answer 1322 session for a particular user,but rather utilize previous user 1314 interactions with the system toprovide decisions; asking questions 1320 and answers 1322 to a user 1314to allow the system to learn in association with future decisions, butproviding rewards to the user 1314 rather than decisions; askingquestion 1320 and answers and making a decision 1310 without anylearning, such as simply filtering down results; utilizing the presentinvention's ability to learn how to make a better decision, butproviding that capability to an expert system rather than to usersthrough a question 1320 and answer 1322 interface; and the like. Inembodiments, the system may provide for all the elements of a question1320 and answer 1322 user 1314 session, providing decisions to the user,and learning how to improve decisions.

In embodiments, a user 1314 entering a question 1320 may optionallyspecify dependencies and importances for the question. Dependencies maycontrol when the question can be asked. Importances may specify relativeimportances between different questions 1320 for weighing a user's 1314answers. If the system has to make trade-offs because no one decision1310 result matches all of the answers 1322 specified by a user 1314,the system may try to recommend decision results that match highimportance questions over lower importance questions. The system mayalso prioritize asking high importance questions over low importancequestions. For example, a user 1314 entering a new question like “Wherein the United States do you want to vacation” set a dependency thatrequires an existing question such as “Where in the world do you want togo” to have been answered with “The United States” before the newquestion “Where in the United States do you want to vacation” can beasked.

In embodiments, the present invention may provide for a system with auser 1314 interface through which the user 1314 may interact with thefacilities of the system. The system may include several parts, some ofwhich may be the website, the supervisor, and a collection of widgets.Widgets may be collections of code that collect, process, and render asingle piece of content on the website. The website may consist ofinterfaces for end-users, staff members, and registered users to getdecisions, edit the decisions, and view reports on system performance.The supervisor may be a container for running widgets so that a widgetcan perform time-consuming data collection and processing ahead of user1314 requests to render that content.

For example, a widget might collect videos about decisions from theinternet. The widget, in the supervisor, might crawl the web looking forvideos about each decision 1310 and store videos it finds in a database.When the user 1314 comes to the website and gets a particular decision,the website may ask the video widget to render itself and display anyvideos it has previously found.

A plurality of instances of the supervisor may be running on multiplecomputers in order to scale up the widget's processing. Each widget maybe running on its own computer. Similarly, many computers may beproviding interfaces to the system through web-servers, instantmessaging, voice gateways, email, programmatic APIs, via being embeddedin third party websites, and the like.

In embodiments, attributes may be combinations of a question 1320 andone particular answer 1322 to that question. For example, if a question1320 was “How old are you?” and the answers to that question 1320 were“under 18”, “20-30” and “over 30”, then an attribute would be “How oldare you? Under 18”. The system may work by learning the relationshipbetween attributes and decisions. When the system asks a question 1320and the user 1314 gives an answer 1322 then the system may take thatattribute and see which decisions are associated with it.

In embodiments, the system may understand that some attributes representcontinuous values while others represent discrete values. When usingcontinuous attributes, the system may be able to make more intelligenttradeoffs such as understanding that it is frequently acceptable torecommend a product that costs less than the user 1314 asked for butrarely acceptable to offer a product that costs more than the user 1314asked for.

In embodiments, the relationships between attributes and decisions maybe learned from users, explicitly given to the system or somecombination of the two, and the like. For example, a price attribute of“How much do you want to spend? Under $200” might be explicitly linkedto cameras that fall into that price range based on data from experts,ecommerce sites/APIs, etc. The relationship between the attribute “Howwill you use the camera? On vacations” and possible vacationdestinations might be fully learned however.

When entering new questions 1320, answers 1322, and results the user1314 may optionally specify the relationships between attributes anddecision results. For example, if a user 1314 were to enter the question“how much do you want to spend?” in the “which camera should I buy”topic, the user 1314 may also specify to the system that the answer“under $200” should be associated with cameras X and Y but not camera Z.Then, if a future user were to use the “which camera should I buy” topicand were to answer the “how much do you want to spend” question with theanswer “under $200” that user 1314 may have a higher chance of beingrecommended camera X and Y over camera Z.

After seeking advice from the system and receiving a decision result, auser 1314 may also be given reasons from the system as to why thatparticular decision result was recommended. This explanation may alsoallow the user 1314 to change the attributes for the decision result ifthe user 1314 believes that the decision result was recommended in errorby the system.

In general, the relationships learned may involve training from users,experts, employees, automated data feeds from third parties, or somecombination.

In embodiments, there may be various ways that the system can recommenda solution and select the next question 1320 to ask the user. Possiblemachine learning systems may be geometric systems like nearest neighborsand support vector machines, probabilistic systems, evolutionary systemslike genetic algorithms, decision trees, neural networks, associatedwith decision trees, Bayesian inference, random forests, boosting,logistic regression, faceted navigation, query refinement, queryexpansion, singular value decomposition and the like. These systems maybe based around learning from complete game plays (e.g., all attributesgiven by a user 1314 before getting a decision), the answers toindividual questions/subsets of game plays, only positive feedbacks,only negative feedbacks or some combination of the two. Additionally,the system may take into account previous interactions the user 1314 hadsuch as remembering previously answered questions, decisions that theuser 1314 liked or did not like, which areas of advice the user 1314previously sought advice in, etc. Additionally, the system may take intoaccount factors that are implicitly provided by the user 1314 such astime of day and date the user 1314 used the system, the user's IPaddress, client type (e.g., Firefox, IE, cell phone, SMS, and the like),and other such data.

In embodiments, the present invention may provide for a machine learningsystem that goes well beyond the capabilities of collaborativefiltering, such as through explicitly asking questions 1320 instead ofimplicitly learning based on a user's behavior, which may be much morepowerful since the system is not left trying to infer the user's intent,mood, etc. Also, choosing the questions 1320 to ask the user 1314 basedon what they've already answered may allow the present invention to zeroin on nuances that would otherwise be missed. The present invention mayhave the ability to explain decisions, such as providing decisionsbeyond simple extrapolations form past behavior such as in, ‘otherpeople who bought X, Y and Z also liked product A’. Instead, the presentinvention may be able to say the user 1314 should ‘do A because the user1314 said they wanted X, liked Y and believed Z’. In addition, thepresent invention may allow users to contribute new questions 1320 thatmay be useful, and then automatically learn under which contexts, ifany, the question 1320 is helpful. In another area of difference, thepresent invention's machine learning technology may be able to providedecisions in a great variety of user 1314 interest areas, wherecollaborative filtering has difficulties being applied tonon-product/media applications. For instance, collaborative filteringwould not be easily applied to helping a user 1314 make a decision 1310on a highly personal topic, such as whether they should get a tattoo, ora rare question 1320 such as whether a particular expense can bededucted on the user's tax return. The present invention may be capableof such applications. In embodiments, the present invention may be ableto use pre-programmed expert advice inter-mixed with advice learned froma group of users to make decisions to users.

In embodiments, the system may have a wiki web interface for editing allof the data on the system. The web interface may be used toedit/create/delete questions, answers, attributes, and solutions. Eachsolution may also have a variety of information associated with it,which may be shown on the decision page when that solution isrecommended. For example, when recommending a vacation in Cancun therecommendation page might show videos about Cancun. All of thisancillary data about the solution may also be editable through the wiki.

In embodiments, the wiki may be used to edit data collected by widgetsrunning in the supervisor. This may allow the widgets to collect dataahead of time and then have a human quality assurance process to reviewand change the collected data.

In embodiments, the system may maintain a history of all changes made byeither the widgets or humans. For example, one use of this history maybe to review the work done by hired contractors doing content qualityassurance. Another use of this history may be making sure that thewidgets do not undo work done by humans. For example, if the widgetscollect a particular video and a human deletes that video because it isinappropriate, then the widget can use the history to not re-add thatvideo again sometime in the future. Finally, if data is corrupted orincorrectly deleted the history may allow a means of recovery.

In embodiments, when widgets find new content they may queue tasks to ahuman workflow for validating and editing that content.

In embodiments, in order to learn, the system may sometimes make randomor semi-random decisions in hopes of recommending something that thesystem wouldn't have expected to be useful, but which may turn out to beuseful. If the system wants to use what it has already learned, then itmay not make random choices in which questions 1320 it asks and whichdecision 1310 it makes. There may be a tradeoff between using what isalready known, also referred to as exploitation, and potentiallylearning something new, also referred to as exploration. Exploitationmay lead to a more satisfied user, while exploration may make the systemsmarter.

In embodiments, one way to make this trade-off when selecting questions1320 to ask the user 1314 may be to ask questions 1320 that the systemis confident are useful in making a decision 1310 and then picking a fewrandom questions 1320 to ask. Another way to make the trade-off may beto have a fixed budget in every user 1314 interaction where a fixed setof questions 1320 are based on exploitation and the next set are basedon exploration.

In embodiments, decisions may also be explored or exploited. If thesystem wants to learn, it may show a random decision. Instead of showinga purely random decision, the system may also show a decision 1310 thatmeets some requirements specified by the user 1314 and is purelyexploring within the remaining requirements. For example, instead ofpicking a random camera to show the user 1314 the system could pick arandom camera that meets the user's price requirements. This may resultin more efficient training since the system may be less likely to show adecision 1310 that has no chance of meeting the user's needs. Ratherthan showing a random decision 1310 when exploring, the system may alsoshow both the exploited decision 1310 and an explored solution and getfeedback on each separately from the user. Alternatively, the systemcould inject a limited amount of randomness and pick a decision “like”what the system's best guess is. For example, the system may predictthat the user 1314 will like one particular camera but could insteadrecommend another similar but not identical camera in order to balancemaking a reasonable decision 1310 and still learning new informationfrom the user. In embodiments, the system may identify to the user 1314when it is asking questions 1320 or making decisions through explorationvs. exploitation, or it may not.

In embodiments, the system may be viewed as surveying users about thevarious things it is recommending. For example, the system may ask theuser 10 questions 1320 about the Canon SD1000 camera. This may provide arich set of data about each camera allowing the system to start buildinglists of what kind of user 1314 is likely to like this camera. Thesystem may build a ranked list of decisions for each attribute, such asfrom most likely to be liked to least likely to be liked, given thatattribute. For example, the system may build a list of cameras in orderlikely to be liked by people who say “How old are you? Over 50”. Thismay be shown by the system as the top 10 cameras for users over 50.Numerous of these top 10 lists may be constructed based on the system'sdata. These lists may also be combined to form new lists. For example,given the ranked lists of cameras for the attribute “How old are you?Over 50” and another list for the attribute “Why are you buying acamera? Travel”, the system may construct a new ranked list of camerasfor the “Over 50 year old users who want a travel camera”. Thesecombinations of top lists may be pre-generated, generated on-demand byincrementally asking the user 1314 to select new top lists, and thelike.

In embodiments, these “top lists” may be used for a variety of purposes.Some users may not want to answer a series of questions 1320 beforereceiving a decision. Instead, they may be able to browse through theselists and find a relevant decision. The system may have a large numberof top lists, such as thousands or tens of thousands, each of which mayhave their own web page. In addition, these pages may contain a largeamount of content that may be indexed by search engines and bring usersto the system's website. Alternatively, users 1314 may use a searchinterface in the system itself to find the area of advice they want adecision in. Various top lists may be used to short-cut the dialog byimplicitly answering some of the questions 1320 in the dialog based onthe toplist. For example, there could be an area of advice called“vacations” and a top list called “romantic honeymoon vacations inItaly” that servers as a short cut or gateway into the “vacations” topicwith several questions 1320 from the “vacations” dialog alreadyanswered: “Where do you want to go? Europe”, “Where in Europe do youwant to go? Italy”, “Are you traveling on a special occasion? Yes”,“What is the special occasion? honeymoon”. This may serve as analternate interface for the user 1314 to seek advice through atraditional search interface without engaging in a question and answerdialog.

In embodiments, various pages on the site may have self-containeddisplays of information called widgets. For example, the decision pagesmay have a widget that shows how other people who liked this question1320 answered various questions, videos/pictures about the decision,links to other web sites that have information about the decision,personalized pros and cons of this decision 1310 based on how the user1314 answered questions, lists of other decisions that a similar, listsof other decisions that would have been made had questions 1320 beenanswered differently, lists of awards/honors for this decision (such asConsume Reports recommended), and the like.

In embodiments, the system may allow users to navigate through theuniverse of decisions (e.g., cameras, vacation destinations, etc) alongdimensions that are not commonly available. For example, instead ofbeing shown a camera and only letting the user 1314 say “show memore/less expensive cameras” the system may let the user 1314 say “showme cameras that are more liked by young people”, “show me a camera thatis better for travel and less stylish”, and the like. Dimensions like“style”, “good for travel”, “bad for young people”, and the like, may begenerated as a side-effect by asking users questions 1320 and thenlearning what is a good decision 1310 given those answers.

In embodiments, navigating along alternative dimensions may be used as astarting point for the user 1314, instead of the user 1314 selecting anarea to seek advice in and then engaging in a dialog. The user 1314 maystart interacting with the system by using a search interface or anexternal search engine to search for a specific decision result, such asa product name or travel destination. The system would then show theuser information about that specific decision result and allow the user1314 to navigate to other decision results, engage in a dialog to refinewhat the user 1314 is looking for, or show the user 1314 informationthat the system has learned (through machine learning, expert advice orsome combination) about this specific decision result. For example, auser 1314 may use a search interface to navigate to a web page showinginformation on a Canon SD1100 camera. The system may show other camerasthat people looking for a Canon SD1100 also like, allow the user 1314 tofind similar cameras along non-traditional feature dimensions such as acamera that is better for taking pictures of sporting events, as well asshow what the system knows about the Canon SD1100 such as “great fortravel”, “not good for people learning photography”, “Available forunder $200”, “Preferred by people who are consider themselves designconscious”, and the like.

In embodiments, another possible interface may be to show users a listof decisions and display a simple explanation for why each decision 1310is being made. For example, when recommending cameras the system mayshow three cameras and say that one is “cheaper”, one has “longer zoom”and the other is “better for travel”. This may help the user 1314 seealternatives that they may not have otherwise seen based on how theyanswered the questions 1320 leading up to the decision 1310.

In embodiments, users may be asked different types of questions, such asquestions 1320 about the item being recommended (price, color, etc),questions 1320 about themselves, and the like. The system maydifferentiate users along dimensions, such as psychographic dimensions,demographic dimensions, and the like. Properties of users that may bepredictive may include the user's age, sex, marital status, whether theylive in rural/urban areas, frequencies of church attendance, politicalaffiliation, aesthetic preferences, sense of irony/sense of humor,socio-economic background, taste, preference for neat or disorganizedlifestyle, degree to which they plan ahead of time, and the like.

In embodiments, it may be difficult to directly ask questions 1320 andinstead the system may try to measure things that are correlatedinstead. For example, instead of asking about income, the system mightask where the user 1314 prefers to shop (e.g., Wal-Mart, Target, Saks,etc). Aesthetics may be determined via showing pictures of art, livingrooms, clothes, and the like, and asking which style the user 1314prefers. In embodiments, pictures may take the place of the question(and the answers may be about how you react to the picture) or thepicture can take the place of answers to questions 1320 such as “Whichof the following best resembles the clothes you like to wear”.

In embodiments, the system may group questions 1320 by whether they areabout the item being recommended or about the user. The system mayexplain what type of questions 1320 it is asking in order to help theuser 1314 understand the value of otherwise surprising and potentiallyoffensive questions 1320 being asked. The system may also display othertypes of messages to the user 1314 while asking questions, such astelling the user 1314 how many questions 1320 remain, taunting the user1314 by saying the system can already guess what decision 1310 to make,and the like.

In embodiments, instant messenger (IM) systems may provide a naturalinterface to the question 1320 and answer 1322 dialog of the system. Forexample, a user 1314 may invite our system to their “buddy list” andthen initiate a dialog to get a decision 1310 over IM. The system may IMthe first question 1320 to the user, the user 1314 may then IM theiranswer 1322 back, and the like, until eventually the system IM'ed theuser 1314 a link to the decision, or directly IM'ed the name of thedecision 1310 to the user. In embodiments, other forms of communicationsmay also be used, such as cell phones, SMS, email, and the like.

In embodiments, the system, such as in the form of an application, maybe embedded in third party web sites. For example, the system could beput on a website that sells cameras and offer to recommend relevantcameras to the user. Alternatively, after the user 1314 searched forcameras and had a list of potential cameras they were interested in, thesystem could ask questions 1320 to help the user 1314 decide amongst thelist of cameras. For example, if all of the cameras that the user 1314was considering were good for travel the system would not ask about howthe user 1314 wanted to use the camera, but the system might realizethat asking whether interchangeable lenses were desired could be used torecommend one camera over another.

In embodiments, the system may make decisions in a plurality of topicareas, such as: products (e.g., cameras, TVs, GPS/navigation, homeaudio, laptops, bath & beauty, baby, garden/outdoor, automobiles,jewelry, watches, apparel, shoes, and the like), travel (e.g., where togo, where to stay, what region to visit, what to do there, and thelike), financial (e.g., which mortgage, whether to refinance, whichcredit card, whether something is deductible on taxes, what type of IRAto save in, asset allocation for investments, and the like), gifts forvarious holidays and occasions, other date-based decisions (what todress up for Halloween, and the like), personality (e.g., about a user'spersonality, about their relationships, their career, and the like),recommending the right pet, drinks and other aspects of night-life,books, movies, film, /music, concerts, TV shows, video games, where toeat, what to order, celebrity related such as which celebrity the user1314 is most similar to, recommending a gift, what neighborhood to livein, what to watch on television, and the like.

In embodiments, the system may be used to diagnose problems, such as inthe areas of technology/IT (e.g., computer, software, printers, homenetworking, wireless, business networks, performance issues, and thelike), medical/health, automotive, relationship or interpersonalproblems, home and building problems, and the like.

In embodiments, users of the system may be either anonymous or logged inusers. A logged in user 1314 may be one that has created an account onthe site. Logged in users may also have profile pages about them.Content on the profile page may include basic information about thatuser (nickname, picture, etc), decisions they have received and liked,decisions the system predicts the user 1314 will like even though theuser 1314 has not answered questions 1320 in that topic area, lists offacts about the user 1314 that the user 1314 has given so that they donot need to be repeated each time the user 1314 uses the system for adecision (e.g., the user's age or their aesthetic preferences can begiven once and remembered across different times the user uses thesystem), lists of tasks that the system thinks the user 1314 may bequalified and interested in doing via the wiki (such as reviewing newuser 1314 submitted content, fixing spelling errors in user 1314submitted content, reviewing new content found by the widgets, etc),other users with similar answers to questions, and the like.

In embodiments, users may also have various titles, ranks or levelswhich may affect what they can do on the system. For example, some usersmay be given the title of “moderator” in a particular topic which wouldallow those users to edit certain aspects of those topics. The ranks andtitles may be assigned manually or by through automatic means includingbeing based on how many decisions they have given, how many newquestions 1320 or solutions they have contributed to the system, howmany tasks they have accomplished using the wiki, how well they answer1322 certain questions 1320 in the various topics, and the like.

In embodiments, non logged in users may not have the benefit of usingthe system with a large selection of aesthetic or taste-basedpreferences already entered into their profiles. Based on learning ormanual training from logged in users 1314, the system may select someaesthetic questions to ask in question dialogs when non-logged in usersseek advice in particular topic areas. For example, based on logged inusers answering taste questions about themselves and then givingfeedback about which cars they like and don't like, the system may learnthat a question 1320 about whether the user enjoys gourmet dining isuseful to ask non-logged in users trying to decide between a Toyota anda Lexus. Using the attribute associations learned or manually specifiedby logged in users, the system may then adjust whether it recommends theToyota or the Lexus to the non-logged in user.

In embodiments, the system may learn from users submitting feedback ondecisions. Some users may either intentionally or unintentionally giveincorrect feedback. For example, a vendor may try to game the system tomake their product be highly recommended. Alternatively, a user 1314 whodoes not know much about video games may recommend a video game that inreality is not a good video game. The system may try to filter outfeedback from these users by a variety of means. The system may throttlethe number of feedbacks that a given user 1314 can submit (and have ahigher throttle limit if the user 1314 is logged in or has a highrank/title). The system may also throttle or weight feedback based onhow well the user 1314 answers certain ‘test’ questions 1320 during thequestion 1320 & answer 1322 phase in order to test the user's knowledgeof the subject and weigh feedback from knowledgeable users more thanunknowledgeable users. The system may also require the user 1314 to passa ‘captcha’ (Completely Automated Public Turing test to tell Computersand Humans Apart) before their feedback is counted or they get adecision. The system may also look at the series of answers given by theuser 1314 and weight the user's feedback based on that series ofanswers. For example, if the user 1314 either always clicked the firstanswer 1322 or the user 1314 clicked in a very improbable way, then thesystem may weight that user's feedback lower. Finally, the system maychange the weight of the user's feedback or decide to not show adecision 1310 based on the history of previous game plays. For example,the 10th time a user 1314 tries to get a camera decision 1310 the systemmay weight their feedback less than on the 9th time.

In embodiments, the system may include search engine optimization (SEO),the process of improving the system's website rankings within majorsearch engines. This process may be broken down into severalmostly-automated steps, such as discovering the keywords that users aresearching for, understanding the competition in the search engines tohave the site's page come up when users search for these words,understanding how search engines rank sites, understanding what changesto the system's website need to be made in order to increase the site'sranking for common searches, and the like.

In embodiments, discovering keywords that users may be searching for maybe found through different means, such as using keyword suggestion toolssuch as what Google and Yahoo provide, using data about historicalsearches licensed from third party data providers and crawling otherwebsites to see what words they use, and the like. Once these keywordsare found, the system may use the data in many ways, such as bidding onthose words via search engine marketing (SEM), developing content on thesystem's site about those keywords in hopes of getting search traffic inthe future, looking at how our competitors are using those samekeywords, and the like.

In embodiments, the system may understand what other sites are doing andhow they rank in the search engines by running keywords through thesearch engines and looking at who is advertising on each keyword andwhat the top natural search results are for each keyword. The sitesdiscovered through this process may be crawled to discover morepotential keywords. The system may also decide to develop new content oravoid a market based on this competitive information. If there are fewhighly ranked sites in a content area, the system may develop content inthat area.

In embodiments, the system may understand that paid advertisements thatbring users 1314 to the site are relatively cheap in one topic area ofadvice on the site and expensive in another. The system may thereforetry to advertise for the low-cost traffic, help those users 1314 withtheir decision, and then recommend that those users 1314 use the systemin a topic area that is expensive to advertise and buy traffic in. Forexample, the system may run ads for people who want to figure out whatdog breed they should buy, help those users 1314 decide what dog breedis right for them, and then direct them to figure out where they shouldbuy their pet medicines. The latter topic area being one that may beexpensive for the system to source traffic in due to expensive ad rates,while the former topic area may be relatively cheap, as few existingbusinesses may be competing for customers who want advice on what typeof dog to get.

In embodiments, the system may understand how search engines rank theirnatural (non-sponsored) search results by studying the relationshipbetween sites that come up when a search is done and factors of thosesites. Possible factors that may be correlated between sites that comeup with high ranking may be factors such as the content of the site,number and quality of other sites linking to the site, the type ofcontent on those other linking sites, and the like. From the prior step,the system may generate a list of site factors, ranked by their abilityto increase a sites ranking in the search engines, and the like. Thesystem may then use this ranked list to make changes to the site toincrease the probability that the site as a whole, or certain pages onthe site, will be highly ranked in the search engines.

Search engines may typically utilize a keyword index to find documentsrelevant to a user's query. In embodiments, the present invention mayutilize a “decision index”, which may also map user-input to relevantdocuments. The index may be built automatically, experts may hand buildthe index, the index may be learned through feedback from differenttypes of users who implicitly or explicitly decide to train the system,and the like. The results of the search utilizing the decision index,may be displayed as a list of documents, a single document, and thelike.

Referring to FIG. 1, an embodiment for a list of topics 102 in thesystem from which users may get decisions is presented, includingcameras, cell phones, coffee and espresso, drinks, favorite celebrity,GPS devices, grills, Halloween, laptops, personality, toe rings, TVs,vacations, video games, watches, and the like. In addition, there may bean indicator as to the number of decisions learned 104 from userratings, such as learned from 43,921 user ratings.

Referring to FIG. 2, an embodiment of an example question 1320 that thesystem may ask a user 1314 is provided. In this example the user 1314 isasking for a decision 1310 related to the purchase of a camera, and thequestion 1320 is “How much are you willing to spend?” The user 1314 maynow choose from the selection 204, such as to select between less than$200, up to $300, up to $500, more than $500, I don't know, and thelike. In addition, there may be an indication as to how many questions1320 may be asked 202, such as in “In 10 questions or less, get cameradecisions preferred by people like you.” In embodiments, the user mayalso offer their own question, their own answer, their own decision, andthe like, where the system may utilize this information in the currentor future decision session. In embodiments, the user 1314 may choose toskip the question 208, where the user 1314 may now be provided analternate decision based on a reduced amount of information availablefrom the user, the system may ask the user alternate questions 1320 tomake up for the skipped question 208, the question 1320 may have been atest question and will not affect the resulting decision 1310, and thelike.

Referring to FIG. 3, an embodiment of an example picture question 1320that the system may ask a user 1314. In this example, the system may beasking a question 1320 whose answer 1322 may better enable the system todetermine a personal characteristic of the user 1314. For instance, thequestion 1320 as illustrated asks “Which of these causes you the mostconcern?”, where the picture choices 304 are indicative of certaintopics, such as pollution, finances, national defense, health, and thelike. This question 1320 may be targeted to the current user or beinserted as an experimental question. In embodiments, the user 1314 maybe informed that the question 1320 is an experimental question 302, suchas shown in FIG. 3 with the header that reads, “Finally, please answerthe experimental questions submitted by another user.”

Referring to FIG. 4, an embodiment of an example of the type ofinformation 402 the system may show the user 1314 when making aparticular decision 1310 is presented. For example, the decision 1310may be for a certain camera, where information is provided about thecamera, such as a description, who uses it, the best cost for thecamera, how it compares 404 to other cameras, and the like. Inembodiments, other decisions 1310 may be provided, such as with arelative ranking 408, by a score, by a percentage matching, and thelike. The user 1314 may also be queried for feedback 1312, such as beingasked if the decision 1310 is a good decision. In addition, the user1314 may be provided with the opportunity to find out more about thedecision 1310, such as more about the product 410, best price finder412, websites to more advice, and the like.

Referring to FIG. 5 and FIG. 6, the user 1314 may be provided withvarious top lists 502 associated with a topic as described herein, suchas presented in association with a decision, in association with auser's request to view top lists, and the like.

In embodiments, the present invention may provide users with a home page700 including user 1314 identification 702, personal representation,past decisions made, future topics for consideration, decision 1310 tomake today 714, and the like. FIG. 7 provides an example of a user homepage 700, such as what the user 1314 sees when they are logged into thesystem account. Here, there may be a display of recent decisions thesystem recommended, lists of popular topics 708 to get decisions in, asearch interface 710 to find topics, status updates about the user 1314getting benefits for contributing to the system, recent activity 704,access to the user's profile 712, and the like.

FIGS. 8 and 8A provide an example of a user's profile 712 page showinginformation about them and their account. The user 1314 may manage userinformation 802, such as a user's email address, password, and the like.They may also answer questions 1320 about themselves and have theseanswers remembered 810 and automatically used when they use decisionmaking topics in the system. The user 1314 may also receive rewards 804,such as “badges”, and see them displayed as received in response tohelping other users, contributing to the system, and the like. Some ofthese rewards may be based on the quality of the user's contributions,on the quantity of contributions, and the like. In addition, users maybe assigned a demographic group 808 of people who answered questions1320 about themselves similarly.

In embodiments, users may be able to decide they want to contributeexpertise 902 to the system, such as in a ‘teach the system’ mode. FIG.9 shows an example of various links/pages that may allow a user 1314 tocontribute, such as giving the system training about various decisions,rating the quality of pictures and user-contributed prose, findingduplicate items and questions, contributing new decision making topics,contributing new questions 1320 to existing topics, and the like.

In embodiments, the user, after choosing a topic for the system to makedecisions for, may be asked questions. FIG. 10 provides an example ofhow a question 1320 may be presented 1000 to the user. As shown, thepresentation of the question 1320 to the user 1314 may provide differentelements, such as a topic heading 1002, a picture or illustrationassociated with the topic 1004, a question, a set of answer choices, andthe like.

After answering questions, the user 1314 may be provided an answer 1322or decision 1310 associated with the user's original question. FIGS. 11and 11A show an example of how a decision 1310 may be presented 1100 tothe user, and may include a primary decision, information summarizingthe decision, alternate decisions, variations on the decision, and thelike. In addition, the user 1314 may be provided with an opportunity toprovide feedback 1312 to the system, such as whether the user 1314agrees with the decision 1310 or not. The user 1314 may also be providedother suggested topics 1102, such as based on the current topic, answersprovide, history of answers, a user's profile, a user's history ofquestions, topics that other users found helpful, and the like.

FIG. 12 shows an example list of decisions 1200 in a topic. For aproduct topic, such as shown, the “decisions” may be what product tobuy. For other topics, the decision 1310 might be “yes, dump him” or“no, don't get a tattoo”. The decisions may be ranked and ordered basedon their relevancy to the user, based on how the user 1314 answeredquestions, based on how the user 1314 answered questions 1320 in thetopic, and the like. Additionally, the items may be ranked by price, byname, and the like.

FIG. 16 shows an example of a contributor/expert interface home page1600, showing recent contributions to the system 1602 and other usersmaking contributions 1604. In upper right corner is a question forlearning the user's taste preferences 1608.

FIG. 17 shows an example of a question in a dialog with the systemasking an objective question 1700, and in this instance, to a userlooking for help deciding what to name their new puppy.

FIG. 18 shows an example of a decision result showing the particularrecommended decision 1800 (in this instance, name your dog Rusty),reviews about this decision from other users 1802 (where it may beranked, such as by their similarity to the user), yes/no buttons 1804(such as for receiving feedback on this decision, showing other decisionareas that the user might enjoy under, and the like), suggested Topics1808. In this example, the system's second and third best recommendeddecisions are listed under the #2 tab 1810 and #3 tab 1812. The systemmay also be engaging in exploration by also recommending a “wild card”decision which may be a decision that was partly picked throughrandomness. The Suggested Topics 1808 may be selected based on howrelevant the system thinks these topics may be for the user and/or howmuch profit the system thinks it may be able to generate from the userusing these other decision areas.

FIG. 19 shows an example of an interface for users 1900 to set theassociations between attributes and decision results. In this example,the decision result “Rusty” should be associated with the attribute “Isthis name for a female, or male dog? Male”.

FIG. 20 shows an example of how a user may edit content in the system2000. In this example the user is able to edit a decision result: it'sname, description, URL for getting more information, etc.

FIG. 21 shows an example of how content that is editable by users mayalso have an interface for seeing prior revisions 2100 to the contentand showing the changes between two prior revisions. Users may alsorevert the changes made by other users if those changes are deemed to beirrelevant or unhelpful. In this case the example shows the differencebetween two revisions to a decision result where the description of theresult has been changed.

FIG. 22 shows an example showing a question being edited by a user 2200.New answers may be added, existing answers re-ordered, the question andanswer text itself edited, and the like. Questions may be optionally“locked” to prevent other users from changing them, such as by indicatedby the pad lock icon 2202.

FIG. 23 shows an example showing that edits to attributes may haverevision histories like other editable content 2300. This example showsthe difference between two revisions of the attribute associationsbetween the decision result “Rusty” and the attributes “How manysyllables do you want the name to have? No more than 2 or 3 or more isOK”.

FIG. 24 shows an example showing a ‘workshop’ screen 2400 where newlyadded areas of advice may be first displayed. In embodiments, expertusers may make additions here without regular users seeing theworks-in-progress. Content that is deemed objectionable, irrelevant, orlow quality may be voted on and removed from the system.

FIG. 25 shows an example showing the system asking the usertaste/subjective questions 2500 in order to learn taste and subjectivepreferences from the user. After answering these questions the systemmay show statistics on how other users answered the same question.

FIG. 26 shows an example of an activity feed 2600 of recent activity bycontributors across the site showing newly added content and experttraining.

In embodiments, the present invention may provide a facility forproviding an improved way to provide decisions to a user 1314 with aquestion 1320 across a broad category of topics, including products,personal, health, business, political, educational, entertainment, theenvironment, and the like. For example, the system may provide decisionson everything from whether a user 1314 should break up with theirboyfriend, to whether you should get a tattoo or not, to whether you candeduct something on your taxes in addition to product decisions, and thelike. In embodiments, the system may provide decisions on any interest auser 1314 may have.

In embodiments, the present invention may provide a decision system thatis flexible and is capable of changing and growing. This may be partlyenabled by the system's use of a dialog of questions 1320 and answers tomake a decision, and then getting feedback from the user 1314 so thesystem can improve. In embodiments, this approach may be significantlymore powerful since the system may ask any question 1320 and thereforeget much better information from the user 1314 about their wants. Inaddition, users may be able to extend the system by entering their ownquestions 1320 and answers for the system to ask, entering in newdecisions for the system to make, and the like. The system may thenautomatically try out newly entered information to see if it is usefulor helpful and use this new information to determine if it is useful,and possibly stop asking/using the questions/decisions that may not beas helpful to users. In embodiments, this approach may provide forbuilding a wisdom-of-the-crowds based decision making expert system forpotentially any topic.

In embodiments, the present invention may also provide improved decisionfacility to the user 1314 by providing decisions by ranking acrossnon-traditional feature dimensions. For example, instead of just rankingcameras by price or size, the system may rank cameras based on how muchthey're liked by retired people, how sexy they look, and the like. Thesystem may then help users navigate across these dimensions. Forinstance, instead of users just being able to say “I like this camera,but want a cheaper one” the system may let them do things like say “Ilike this camera but want one better for learning photography” or “Ilike this vacation, but want one with a more active social scene”.

In embodiments, the present invention may lend itself to a variety ofdifferent user interfaces, such as a web interface, instant messaging,voice, cell phone, SMS/instant messaging, third party use (e.g. a widgeton a third party, web service sold to a third party), and the like. Forexample, a voice interface may be well suited to the system since theremay be a very limited vocabulary that the system must recognize, such asjust the possible answers to each question. In this way, if the systemcan't understand a user response it may just move on to another question1320 instead of annoying the user 1314 by asking them to repeat theiranswer over and over. In another example, the present invention may beintegrated into a third party site, such as a search for a TV on ane-commerce website, where the present invention is a widget to help theuser 1314 narrow down the results, or using the present invention as awidget in association with a real estate website to build an MLS queryfor the user 1314 to find a house that is a good match for them. Inembodiments, the present invention may provide a user interface, both inregard to a physical interface and in the way questions, answers, anddecisions are presented, that provides the user 1314 with asignificantly improved way to obtain decisions on a great variety oftopics.

In embodiments, the present invention may be integrated into third partyproducts in such as way as to improve the third party's user interfaceand user satisfaction. For example some website services providepredictions through past purchase history. In this case, the presentinvention may be able to explore a user's mood or intent, such asthrough asking explicit questions. In the case of search engines, thepresent invention may be able to detect when the user 1314 is trying tomake a decision 1310 and then start to ask them follow on questions. Inthe case of forum sites, mailing lists, news groups, and the like, thepresent invention may provide improved access to decisions and decisionsthat were made by people similar to the user. For example, the presentinvention may be able to search through all the forum posts to findpeople who are in the same situation as the user, and providing whatdecision 1310 the forum community recommended to them.

In embodiments, the present invention may be able to extend e-commerceweb application user interfaces. For instance, a user 1314 may start aproduct search with a keyword search and then ask questions 1320 tonarrow down the results to the best decision 1310 for the user. Thepresent invention may be able to provide a Q&A interface for picking aproduct once the user 1314 clicks into a category page. For example,after clicking cameras on the website, the user 1314 might see a firstquestion. The present invention may be able to rank products alongdimensions that are based on how users answer questions. For example,cameras might be ranked from best to worst ‘travel camera’ based on howpeople answer 1322 the question “What do you want a camera for?” Answer“Travel” and then whether they give positive or negative feedback to aparticular camera. This may allow the e-commerce website to rank a listof camera keyword search results from best to worst travel cameras.

In embodiments, the present invention may be able to provide an improvedsearch engine capability, such as detecting when a user 1314 is tryingto make a decision 1310 and switching to a Q&A interface, based on thesearch results from a keyword search ask follow up questions 1320 tonarrow down or re-rank the results, ask questions 1320 in order to builda keyword search query or to refine a search query, learn feedback basedon which links a user 1314 clicks after being asked questions, and thelike. In addition, the present invention may implicitly learn about theuser 1314 and alter rankings based on these implicit facts, such as whattime of day they're using the system, where they are in the world, whattype of browser they're using, weather where they are, and the like.

In embodiments, the present invention may be able to provide a way forinformation to be gathered and utilized by users. For instance,Wikipedia is a way for users to contribute information such that the enduser 1314 must, to some extent, self validate the accuracy of theinformation subsequently supplied to them. In a similar fashion, thepresent invention may be able to host a web application that utilizesuser contributed content. For instance, instead of learning what theprices of cameras are, the web application could have users input theprices of cameras and then allow other users to self validate theseclaims. In this way, the scope of the contributed information may beallowed to grow organically as users interact with the system.

In embodiments, some e-commerce applications may provide for productsand/or services that are associated with personal preference, and so maybenefit from the present invention. For instance, there are currentlyseveral movie rental web services, where the user 1314 selects moviesfor delivery to their home through the mail. Decisions are also providedto the user 1314 based on what the user 1314 has selected in the past.However, choosing a movie may involve personal interests at the time ofrental that cannot be determined by past selections, such as mood,intent, weather, are they going to be alone or with someone, theircurrent personal relationships, and the like. These types of interestsmay be explored with the present invention through questioning, and assuch, may provide a much more personalized match to the user's interestsat the time of rental.

In embodiments, local search applications may be improved through theuse of the present invention. For example, if a user 1314 wanted adecision 1310 on where to eat dinner, they might search for “dinner innew york” and find a website with suggestions targeted to the query.This interface however falls short when the user 1314 doesn't have aclear idea as to what keywords to include. For instance, the user 1314might not know the key options for food and might not think to searchfor ‘ethiopian food new york.’ The present invention may have theadvantage of being able to figure out what question 1320 it should askin order to narrow down the possibilities. In embodiments, the presentinvention may be able to aid in the building of a search query.

In embodiments, the present invention may provide for an improved way tomatch up users and experts, users and other knowledge based users, andthe like. For instance, a service may be provided to collect users andexperts on different topics. Users may then come to the web interface ofthe service and enter into a session of Q&A where the best match isdetermined. As a result of the questions, the system may provide adecision, where the profile of the expert or other user 1314 isprovided, and where the user 1314 may be asked if they agree with therecommended individual. In embodiments, the user 1314 may be provided ahome page where previous matches and communications may be kept,forwarded to friends, experts rated, and the like.

In embodiments, the present invention may provide a platform for acommunity based question 1320 and answer 1322 application. For instance,users may post questions 1320 to the system, and other users may beallowed to respond. In such a system, a user 1314 may receive answersfrom a single user, multiple users, an automated system, and the like,where the user 1314 may be able to choose which answer 1322 they feel iscorrect. This answer 1322 may be kept private, posted for others toview, posted as the correct answer, provided to the system, and thelike. In embodiments, the system may use the questions 1320 and answersto further develop the system, provide more accurate answers to users,sort the answers provided to the user, filter the answers provided tothe user, and the like. In addition, users of the system may providefeedback to answers provided by other users, contribute to filteringcriteria for eliminating incorrect answers, and the like.

In embodiments, the present invention may be used as entertainment,through machine learning capabilities as described herein. For instance,a user 1314 may provide an input or think of an idea, such as a topic, akeyword, a category, a question, a feeling, and the like, and the systemmay make a guess as to what it is through a series of questions 1320 andanswers. For example, the user 1314 may think of an object, such asbaseball, and the system may utilize machine learning capabilities, suchas geometric systems, to provide questions 1320 to the user. A typicalquestion 1320 may relate to size, such as ‘is it bigger than a toaster?’These questions 1320 may then be answered by the user, such as throughmultiple choice selection, fill in the blank, true/false, free response,and the like. The system may then continue the question 1320 and answer1322 sequence until it has a guess, and provide this guess to the user.In embodiments, this process may continue for a fixed number ofquestions, a random number of questions, a user 1314 specified number ofquestions, a system determined number of questions, a system specifiednumber of questions, and the like. In embodiments, the system mayprovide the user 1314 with a user interface, such as through theInternet via a website, through a stand-alone computational device,through a mobile computational device, through a phone service, througha voice interface, in association with an instant messaging service,through text messaging, and the like. In embodiments, the system may beprovided to a third party, such as a widget to another website, as anAPI to a third party application, and the like. In embodiments, thepresent invention may use non-neural networks for entertainmentapplications, such as playing games.

In embodiments, the present invention may provide a system to assist inthe discovery of new drugs, where the system may provide an aid in theselection and combination of molecules in creating a new drug. Forexample, the system may ask the user 1314 about information associatedwith chemical parameters, such as solubility, reactivity, toxicity, andthe like, and combine these with questions 1320 to probe the user'sexpertise in recognizing molecular structures. As the question 1320 andanswer 1322 sequence progresses, the system may provide the user 1314with insights as to which molecular structures may be stable andsynthesizable. In embodiments, the process may continue until the user1314 has an improved sense for what molecular combinations may make fora new drug, until the choice of new exploratory routes are available forpresentation to the user, until an new potential drug is identified, andthe like.

In embodiments, the present invention may provide for an image finderapplication, where the user 1314 may be assisted in identifying an imagethat fits some subjective criteria that is not necessarily explicitlyknown to the user. For example, a user 1314 may be involved in thedevelopment of a brochure for a company, where they have the text forthe brochure, but need to select an image to support the ideas andemotions that the text is trying to convey. The user 1314 may in thisinstance have a subjective idea as to what type of photograph may berequired, but not necessarily to the extent that they could specify asearch with keywords. The user 1314 may instead first specify the sourceof the images, such as from a file, a database, a website service, fromGoogle images, from an advertiser image bank, and the like. Then theuser 1314 may be asked a series of questions, or be presented with aseries of images to choose from. The answers and/or selections that theuser 1314 chooses may then be utilized in refining the choices that arenext presented to the user, and from which further questions 1320 and/orimage selections may be provided. In embodiments, this process maycontinue until the user 1314 finds an image to select as the finalimage. Additionally, the system may take the user's ‘final selection’and select a group of other similar images for presentation to the user,at which time the user 1314 may choose to continue the process ofselection refinement.

In embodiments, the present invention may be used in a baby namingapplication, where the user 1314 may have only a vague sense of whatnames they might prefer. The user 1314 may be initially asked differenttypes of questions 1320 intended to provide the system with informationto aid in the learning of the user's preferences, such as questions 1320about family, friends, education, heritage, geographic location, placeof birth, hobbies, books read, movies watched, and the like. The systemmay then continue to learn through the presentation of questions 1320associated with name preference in a plurality of ways, such as ratingname, choosing from a list of names, answering questions 1320 pertainingto name, and the like. In embodiments, this process may continue untilthe user 1314 finds a name to select as the final name. Additionally,the system may take the user's ‘final selection’ and select a group ofother similar names for presentation to the user, at which time the user1314 may choose to continue the process of selection refinement.

In embodiments, the present invention may provide decisions for aplurality of topics including, but not limited to, video games, laptops,vacations, cameras, general personality, drinks, cell phones,televisions, grills, watches, coffee machines, toe rings, Halloween, GPSdevices, hottest celebrity, your personal hero, presidential election,baby toys, blogs, camcorders, cars, which star wars character are you,credit cards, hair care, skin care, sex and the city, should I get atattoo, professions, how much allowance, city to live in, dog breeds,fragrance, New York, neighborhood chooser, software, desktop computers,DVD players and recorders, cigars, charities, Broadway shows, speakers,home theater systems, MP3 players, computer networking devices,headphones, memory cards, magazines, books, Oprah picks, books, The NewYork Times bestsellers, business casual clothing, franchises, cookware,toys, toys—educational, athletic apparel, espresso machines, should I goGreek, should I come out to my parents, should I ask for a raise, do Ihave a drinking problem, should I medicate my add/ADHD child, vacuumcleaners, clothes washers and dryers, is working at a startup right forme, humidifiers, are you a good friend, risk of developing diabetes,which foreign language should I learn, microwaves, car audio, what kindof customer are you, wine, should I join the military, which militarybranch should I join, what kind of art will I enjoy, baby and toddlercar seats, baby strollers, baby travel accessories, natural and organicbeauty products, makeup, home audio receivers and amplifiers, copiersand fax machines, printers, breakup with my boyfriend/girlfriend, whichGreek god are you, what game show would I enjoy, computer accessories,which superpower should you have, college, online degree programs,choose a major for college, identity theft prevention, should I hire apersonal trainer, should I buy or lease a car, should I have laser eyesurgery, what should I do about losing my hair, should I start my ownbusiness, should my child start kindergarten, how to entertain my familyvisiting NYC, OTC pain relievers, do I need a living will, miles or cashfor my next flight, best way to whiten my teeth, should I let mydaughter wear makeup, is hypnosis likely to cure my bad habit, EDoptions, sleep aids, OTC allergy pills, how much money to spend on awedding gift, should I buy the extended warranty, is it better to takethe SAT or ACT, personal audio accessories, coffee/espresso drink wouldI enjoy, video game consoles, jeans, downloadable PC games, snacks,vitamins and supplements, which superhero am I, sunglasses, kitchengadgets, pillows, beauty accessories, beauty bags and cases, sportinggoods, which musical instrument is right for me, should I hire adecorator, electronic readers, where do you belong in a shopping mall,power washers, small business, phone system, how much to tip, should Itry Botox, should I get liposuction, risk of skin cancer, should Irefinance my home, car services (NYC), microbrewery beer, gourmetchocolates, am I saving enough for retirement, entertainment centers/TVstands, cookbooks, electric shavers, keep sending nieces/nephews bdaygifts, luggage, computer projectors, energy/workout bars, razors,gourmet ice creams, online dating, newscasts, makeup, tools and brushes,beauty mirrors and compacts, business books, how soon to call after afirst date, places to retire, external hard drives, universal remotecontrols, walking shoes, should I sell my life insurance policy, howgreen are you, do I have an eating disorder, baby cribs, diets and dietbooks, cell phone plans, wedding and engagement rings, am I assertiveenough, does my child play video games too much, tax preparation(personal return), should I get a reverse mortgage, cancel plans withfriends for a date, children's TV shows, kitchen countertops, bathingsupplies, insect repellents, cancer specialist, hospitals, nationalchain restaurants, cereal, should I have kids now, should I hire ananny, movies, beef cuts, target calories per day, do I have OCD, homeair purifiers, auto air fresheners and purifiers, i-phone applications,gay/lesbian vacations, is it ok to ask my co-worker on a date, is mypre-teen ready to babysit, sports/energy drinks, TV shows, officefurniture, motorcycles, reward a child for a good report card, lawntrimmers and edgers, am I too stressed, religion, do you make a goodfirst impression, do you spend too much time online, should I get a newhairstyle, should I home-school my child, diaper bags, should I usecloth or disposable diapers, dog toys, is my partner cheating on me,classic books should my elderly parent stop driving, am I over my ex, isit lust or love, pedometers and heart rate monitors, chewing gum,weather devices, will gas additives help my car, Orlando theme parks,how big of a turkey should I buy, popular music—new releases, selftanner, tax and money management, software, baby bottles and Sippy cups,baby high chairs and booster seats, baby tethers, toasters and toasterovens, comforters sheets and bed linens, flatware sets, pet carriers andkennels, cheese, kitchen faucets, casual shoes, dress shoes, beautyelectronics, am I saving enough for retirement, mutual fund chooser,steak cuts, what is my D&D alignment, acne and pimple medication,bathroom faucets, home exterior lighting, landscape lighting, lawnmowers, aperitif, cognac, gin, rum, scotch, tequila vodkas, whiskeys,Las Vegas shows, sunscreen, running shoes, US MBA programs, patio andoutdoor furniture, kitchen knives, are you a true fan, auto insurance,personal legal services, should I hire a financial advisor, indoor plantselector, delivery services, can I deduct it, pool heaters, sofas, housenumbers, contact lenses, birthday gifts, has my career peaked,electronic books, doorknobs & lock sets, snow removal equipment, greenhome improvement, kids clothing & swimwear, motorcycle helmets, bicyclehelmets, juicers, golf clubs, refrigerators, wine coolers, ranges andovens, air conditioners, Christmas gifts, breakup phrases, cold soremedication, diabetes monitoring devices, smoking cessation, what do I doabout the hair on my back, hormones to counteract menopause, hikingbackpacks, school backpacks, get a website/domain, e-mail services, webhosting, carpets, power tools, tile, water heaters, outdoor paint,window treatments, fireplace screens, indoor lamps, small business legalservices, brunch recipes, ceiling fans, mattresses, Las Vegas hotels andcasinos, salsas, love quiz for valentines, how much to spend on clientgifts, anniversary gifts, outdoors outerwear, casual outerwear, campingtents, sleeping bags, tires, adventure vacations, music downloads, videodownloads, wedding dresses, wedding themes, Manhattan gyms, budget hotelchains, golf courses, ski vacations, US spas, ETF funds, designerhandbags, should I declare bankruptcy, 401k as down payment on home,should I see a psychiatrist, self defense, dishware, dishwashers,political parties, new year's resolutions, cruise lines, familyvacations, baby food, baby health care products, should I shave my head,t-shirts, online photo services, buy a class graduation ring, summerjob/internship, where to volunteer, home alarm systems, diagnose yourrelationship issues, is she/he hot for me, should I adopt, should myaging parents be driving, online bank accounts, BBQ sauces, frozenpizza, recipe finder, should I re-gift it, bodybuilding supplements,home workout equipment, how many hours of sleep do I need, should Iconsider plastic surgery, risk of arthritis, risk of heart disease, riskof osteoporosis, do I have a gambling problem, best dance to learn,bicycles, cat food, dog food, hobby recommender, martial arts, posterart, outdoor flower selector, which Muppet are you, activities for kids,how ethical are you, should I baptize my child, Miami hotels, USnational parks, motor oils, automotive video, blouses, coats, dresses,glasses frames, hosiery, interview clothes, jackets, negligee, pants,shirts, skirts, hats, phones-land lines, steakhouses, which birth methodis right for you, summer camp recommender, march madness bracketchooser, baby formula, New York bakeries, fractional jet ownership, howself confident am I, digital photo frames, do I need an accountant, doesmy child have ADD/ADHD, document shredders, baby monitors, green homeimprovement, conference phones, and the like.

In embodiments, and as depicted in FIG. 13, the present invention mayhelp a user 1314 make a decision 1310 through the use of a machinelearning facility 1302. The process may begin with an initial question1320 being received 1304 by the machine learning facility 1318 from theuser 1314. The user 1314 may then be provided with a dialog 1308consisting of questions 1320 from the machine learning facility 1318 andanswers 1322 provided by the user 1314. The machine learning facility1318 may then provide a decision 1310 to the user based on the dialog1308 and pertaining to the initial question 1304, such as arecommendation, a diagnosis, a conclusion, advice, and the like. Inembodiments, future questions 1320 and decisions 1310 provided by themachine learning facility 1318 may be improved through feedback 1312provided by the user 1314.

In embodiments, the initial question 1304 posed by the user 1314 may bean objective question, a subjective question, and the like. A question1320 may be provided from amongst a broad category of topics, such astopics pertaining to a product, personal information, personal health,economic health, business, politics, education, entertainment, theenvironment, and the like. The questions 1320 may be in the form of amultiple choice question, a yes-no question, a rating, a choice ofimages, a personal question, and the like. The questions 1320 may beabout the user 1314, provided by another user, provided by an expert,and the like. The questions 1320 may be based on a previous answer, suchas from the current dialog 1308 with the user 1314, from a storedprevious dialog 1308 with the user 1314, from a stored previous dialog1308 with another user. The question 1320 may be a pseudo randomquestion, such as a test question, an exploration question 1320 thathelps select a pseudo random decision 1310 on the chance that the pseudorandom decision 1310 turns out to be useful, and the like. The questions1320 may include at least one image as part of the question. Thequestions 1320 may be along psychographic dimensions. In embodiments,the questions 1320 may not be asked directly to the user 1314, butrather determined from contextual information, such as through an IPaddress, the location of the user, the weather at the user's location, adomain name, related to path information, related to a recent download,related to a recent network access, related to a recent file access, andthe like.

In embodiments, the dialog 1308 may continue until the machine learningfacility 1318 develops a high confidence in a reduced set of decisions,such as a reduced set of decisions presented to the user, a singledecision 1310 presented to the user. The decision 1310 provided by themachine learning facility 1318 may be independent of the order ofquestions in the dialog 1308. The decision 1310 may provide an alternatedecision 1310 when at least one question 1320 in the dialog is omitted,where the alternate decision 1310 may be different based on the machinelearning facility 1318 having less information from the user 1314. Thedecision 1310 may display a ranking of decision choices, such as rankingdecisions across non-traditional feature dimensions. The decision 1310may display at least one image related to the decision 1310. Thedecision 1310 may be a pseudo random decision on the chance that thepseudo random decision 1310 turns out to be useful, such as the pseudorandom decision being part of a system of exploration, where the systemof exploration may improve the effectiveness of the system, the machinelearning facility 1318 may learn from exploration, and the like.

In embodiments, the feedback 1312 provided may be related to, or derivedfrom, how the user 1314 answers questions 1320 in the dialog 1308, howthe user 1314 responds to the decision 1310 provided by the machinelearning facility 1318, and the like. In embodiments, the feedback 1312may be solicited from the user 1314.

In embodiments, users 1314 may extend the learning of the machinelearning facility 1318 by entering new information, where the newinformation may be their own topic, question, answer, decision, and thelike. The machine learning facility 1318 may use the new information todetermine whether the new information is helpful to users.

In embodiments, a user interface may be provided for user interactionwith the machine learning facility 1318, such as associated with a webinterface, instant messaging, a voice interface, a cell phone, with SMS,and the like.

In embodiments, the present invention may help a user make a decision1310 through the use of a machine learning facility 1318. The processmay begin with an initial question 1304 being received by the machinelearning facility 1318 from the user 1314, where the initial question1304 may be associated with one of a broad category of topics, such asproduct, personal, health, business, political, educational,entertainment, environment, and the like. The user 1314 may then beprovided with a dialog 1308 consisting of questions 1320 from themachine learning facility 1318 and answers 1322 provided by the user1314. The machine learning facility 1318 may then provide a decision1310 to the user 1314 based on the dialog 1308 and pertaining to theinitial question 1304, such as a recommendation, a diagnosis, aconclusion, advice, and the like. In embodiments, future questions 1320and decisions 1310 provided by the machine learning facility 1318 may beimproved through feedback 1312 provided by the user 1314.

In embodiments, and as depicted in FIG. 14, the present invention mayhelp a user make a decision 1310 through the use of a computing facility1402. The process may begin with an initial question 1304 being receivedby the computing facility 1418 from the user 1314. The user 1314 maythen be provided with a dialog 1408 consisting of questions 1320 fromthe computing facility 1418 and answers 1322 provided by the user 1314.The computing facility 1418 may then provide a decision 1310 to the user1314 based on an aggregated feedback 1428 from the feedback from aplurality of users 1412. In embodiments, the computer facility 1418 mayimprove future questions 1320 and decisions 1310 provided by thecomputing facility 1418 based on receiving feedback 1412 from the user.

In embodiments, the present invention may help a user make a decision1310 through the use of a machine learning facility 1318. The processmay begin with an initial question 1304 being received by the machinelearning facility 1318 from the user 1314. The user 1314 may then beprovided with a dialog 1308 consisting of questions 1320 from themachine learning facility 1318 and answers 1322 provided by the user1314, where the number of questions 1320 and answers 1322 providedthrough the dialog 1308 may determine the quality of the decision 1310.The machine learning facility 1318 may then provide a decision 1310 tothe user based on the dialog 1308 and pertaining to the initial question1304, such as a recommendation, a diagnosis, a conclusion, advice, andthe like. In embodiments, future questions 1320 and decisions 1310provided by the machine learning facility 1318 may be improved throughfeedback 1312 provided by the user. In embodiments, the quality may behigh when the number of questions 1320 and answers 1322 large, such asgreater than 10 questions, greater than 15 questions, greater than 10questions, and the like. In embodiments, the quality may be good qualitywhen the number of questions 1320 and answers 1322 is small, such asless than 10 questions, less than 5 questions, less than 3 questions,one question, and the like.

In embodiments, and as depicted in FIG. 15, the present invention maymake a decision 1310 through the use of a machine learning facility1318. The system may include a machine learning facility 1318 that mayreceive an initial question 1304 from the user 1314, a dialog facility1502 within the machine learning facility 1318 providing the user 1314with questions 1320 and accepting answers 1322 from the user, themachine learning facility 1318 providing a decision 1310 from a decisionfacility 1504 to the user 1314, and the like. In embodiments, thedecision 1310 provided to the user 1314 may be based on the exchange ofdialog 1308 between the user 1314 and the machine learning facility1318, and pertain to the initial question 1304. Further, the machinelearning facility 1318 may receive feedback 1312 through a feedbackfacility 1508 from the user 1314 to improve future questions 1320 anddecisions 1310 provided by the machine learning facility 1318.

In embodiments, the present invention may help a user 1314 make adecision 1310 through the use of a machine learning facility 1318. Theprocess may begin with an initial question 1304 being received by themachine learning facility 1318 from the user 1314 through a third party,such as a search application, a social network application, a serviceprovider, a comparison shopping engine, a media company's webenvironment, and the like. The user 1314 may then be provided with adialog 1308 consisting of questions 1320 from the machine learningfacility 1318 and answers 1322 provided by the user 1314. The machinelearning facility 1318 may then provide a decision 1310 to the user 1314based on the dialog 1308 and pertaining to the initial question 1304,such as a recommendation, a diagnosis, a conclusion, advice, and thelike. In embodiments, future questions 1320 and decisions 1310 providedby the machine learning facility 1318 may be improved through feedback1312 provided by the user 1314.

In embodiments, the present invention may help a user 1314 make adecision 1310 through the use of a machine learning facility 1318. Theprocess may begin with an initial question 1304 being received by themachine learning facility 1318 from the user 1314 through a third partysearch application, where the user 1314 begins with a keyword search onthe third party search application and then is provided a dialog 1308consisting of questions 1320 from the machine learning facility 1318 andanswers 1322 provided by the user 1314. The machine learning facility1318 may then provide a decision 1310 to the user 1314 based on thedialog 1308 and pertaining to the initial question 1304, where thedecision 1310 may be provided back to the third party searchapplication, such as in the form of a sorted list.

In embodiments, the present invention may help a user 1314 make adecision 1310 through the use of a machine learning facility 1318. Theprocess may begin with an initial question 1304 being received by themachine learning facility 1318 from the user 1314. The user 1314 maythen be provided with a dialog 1308 consisting of questions 1320 fromthe machine learning facility 1318 and answers 1322 provided by the user1314, where the machine learning facility 1318 may utilize third partyinformation, functions, utilities, and the like. The machine learningfacility 1318 may then provide a decision 1310 to the user 1314 based onthe dialog 1308 and pertaining to the initial question 1304, such as arecommendation, a diagnosis, a conclusion, advice, and the like. Inembodiments, third party information, functions, utilities, and thelike, may include an application programming interface (API) enablingthe collection of cost information, product information, personalinformation, topical information, and the like.

In embodiments, the present invention may help a user 1314 make adecision 1310 through the use of a machine learning facility 1318. Theprocess may begin with an initial question 1304 being received by themachine learning facility 1318 from the user 1314 through a third partysearch application, where the user 1314 begins with a keyword search onthe third party search application and then is provided a dialog 1308consisting of questions 1320 from the machine learning facility 1318 andanswers 1322 provided by the user 1314 The machine learning facility1318 may then provide a decision 1310 to the user 1314 based on thedialog 1308 and pertaining to the initial question 1304, such as arecommendation, a diagnosis, a conclusion, advice, and the like. Inembodiments, the decision 1310 may be provided back to the third partysearch application based at least in part on collaborative filtering.

In embodiments, the present invention may help a user 1314 make adecision 1310 through the use of a machine learning facility 1318. Theprocess may begin with an initial question 1304 being received by themachine learning facility 1318 from the user 1314. The user 1314 maythen be provided with a dialog 1308 consisting of questions 1320 fromthe machine learning facility 1318 and answers 1322 provided by the user1314. The machine learning facility 1318 may then provide at least oneimage with the decision 1310 to the user 1314 based on the dialog 1308and pertaining to the initial question 1304, such as a recommendation, adiagnosis, a conclusion, advice, and the like. In embodiments, the imagemay be a photograph, a drawing, a video image, an advertisement, and thelike.

In embodiments, the present invention may help a user 1314 make adecision 1310 through the use of a machine learning facility 1318. Theprocess may begin with an initial question 1304 being received by themachine learning facility 1318 from the user 1314. The user 1314 maythen be provided with a dialog 1308 consisting of questions 1320 fromthe machine learning facility 1318 and answers 1322 provided by the user1314 where the questions 1320 may be determined at least in part fromlearning from other users of the machine learning facility 1318. Themachine learning facility 1318 may then provide a decision 1310 to theuser 1314 based on the dialog 1308 and pertaining to the initialquestion 1304, such as a recommendation, a diagnosis, a conclusion,advice, and the like. In embodiments, the decision 1310 may be based atleast in part on learning from decisions 1310 provided by other users ofthe machine learning facility 1318.

In embodiments, the present invention may help a user 1314 make adecision 1310 through the use of a machine learning facility 1318. Theprocess may begin with an initial question 1304 being received by themachine learning facility 1318 from the user 1314. The user 1314 maythen be provided with a dialog 1308 consisting of questions 1320 fromthe machine learning facility 1318 and answers 1322 provided by the user1314. The machine learning facility 1318 may then provide a decision1310 to the user 1314 based on the dialog 1308 and pertaining to theinitial question 1304, such as a recommendation, a diagnosis, aconclusion, advice, and the like. In embodiments, the decision 1310 maybe based at least in part on collaborative filtering.

In embodiments, the present invention may help a user 1314 make adecision 1310 through the use of a machine learning facility 1318. Theprocess may begin with an initial question 1304 being received by themachine learning facility 1318 from the user 1314. The user 1314 maythen be provided with a dialog 1308 consisting of questions 1320 fromthe machine learning facility 1318 and answers 1322 provided by the user1314. The machine learning facility 1318 may then provide a decision1310 to the user 1314 based on the dialog 1308 and pertaining to theinitial question 1304, such as a recommendation, a diagnosis, aconclusion, advice, and the like. In embodiments, the decision 1310 maybe based at least in part on collaborative filtering whose context isprovided through the dialog 1308, such as at least one questionproviding the context for the collaborative filtering.

In embodiments, the present invention may help a user 1314 make adecision 1310 through the use of a machine learning facility 1318. Theprocess may begin with an initial question 1304 being received by themachine learning facility 1318 from the user 1314. The user 1314 maythen be provided with a dialog 1308 consisting of questions 1320 fromthe machine learning facility 1318 and answers 1322 provided by the user1314. The machine learning facility 1318 may then provide a decision1310 to the user 1314 based on the dialog 1308 and pertaining to theinitial question 1304, such as a recommendation, a diagnosis, aconclusion, advice, and the like. In embodiments, the decision 1310 maybe based only on information gathered through a plurality of user 1314sof the machine learning facility 1318 and pertaining to the initialquestion 1304, where at least one of the plurality of user 1314s of themachine learning facility 1318 may be the user 1314 associated with thedialog 1308.

In embodiments, the present invention may help a user 1314 make adecision 1310 through the use of a machine learning facility 1318. Theprocess may begin with an initial question 1304 being received by themachine learning facility 1318 from the user 1314. The user 1314 maythen be provided with a dialog 1308 consisting of questions 1320 fromthe machine learning facility 1318 and answers 1322 provided by the user1314. The machine learning facility 1318 may then provide a decision1310 to the user 1314 based on the dialog 1308 and pertaining to theinitial question 1304, and with limited initial machine learningfacility 1318 knowledge on the subject matter of the initial question1304. In embodiments, the limited initial machine learning facility 1318knowledge may be seed knowledge, may be limited to basic knowledgeassociated with the subject matter of the initial question 1304, may belimited to basic knowledge associated with the subject matter of theinitial question 1304 where the basic knowledge may be expert knowledge.

In embodiments, the present invention may help a user 1314 make adecision 1310 through the use of a machine learning facility 1318. Theprocess may begin with an initial question 1304 being received by themachine learning facility 1318 from the user 1314. The user 1314 maythen be provided with a dialog 1308 consisting of questions 1320 fromthe machine learning facility 1318 and answers 1322 provided by the user1314. The machine learning facility 1318 may then provide a decision1310 to the user 1314 based on the dialog 1308 and pertaining to theinitial question 1304, such as a recommendation, a diagnosis, aconclusion, advice, and the like, where the decision 1310 may be basedon learning from a combination of expert and user inputs.

In embodiments, the present invention may help a user 1314 make adecision 1310 through the use of a machine learning facility 1318. Theprocess may begin with an initial question 1304 being received by themachine learning facility 1318 from the user 1314. The user 1314 maythen be provided with a dialog 1308 consisting of questions 1320 fromthe machine learning facility 1318 and answers 1322 provided by the user1314. The machine learning facility 1318 may then provide acategory-based decision 1310 to the user 1314 based on the dialog 1308and pertaining to the initial question 1304, such as a recommendation, adiagnosis, a conclusion, advice, and the like.

In embodiments, the present invention may help a user 1314 make adecision 1310 through the use of a machine learning facility 1318. Theprocess may begin with an initial question 1304 being received by themachine learning facility 1318 from the user 1314. The user 1314 maythen be provided with a dialog 1308 consisting of questions 1320 fromthe machine learning facility 1318 and answers 1322 provided by the user1314. The machine learning facility 1318 may then provide a decision1310 to the user 1314 where the machine learning facility 1318 mayutilize responses from a plurality of user 1314 s of the machinelearning facility 1318 to categorize and provide decisions 1310 along atleast one of psychographic and demographic dimensions.

In embodiments, the present invention may provide a user 1314 with aresponse through the use of a machine learning facility 1318. The user1314 may be provided with a dialog 1308 consisting of questions 1320from the machine learning facility 1318 and answers 1322 provided by theuser 1314, where the questions 1320 from the machine learning facility1318 may be related to an application, such as an entertainmentapplication, a drug discovery application, a baby name application, andthe like. The machine learning facility 1318 may then provide theresponse to the user 1314 based on the dialog 1308 and pertaining to theinitial question 1304, such as a recommendation, a diagnosis, aconclusion, advice, and the like. In embodiments, future questions 1320and decisions 1310 provided by the machine learning facility 1318 may beimproved through feedback 1312 provided by the user 1314.

The methods and systems described herein may be deployed in part or inwhole through a machine that executes computer software, program codes,and/or instructions on a processor. The present invention may beimplemented as a method on the machine, as a system or apparatus as partof or in relation to the machine, or as a computer program productembodied in a computer readable medium executing on one or more of themachines. The processor may be part of a server, client, networkinfrastructure, mobile computing platform, stationary computingplatform, or other computing platform. A processor may be any kind ofcomputational or processing device capable of executing programinstructions, codes, binary instructions and the like. The processor maybe or include a signal processor, digital processor, embedded processor,microprocessor or any variant such as a co-processor (math co-processor,graphic co-processor, communication co-processor and the like) and thelike that may directly or indirectly facilitate execution of programcode or program instructions stored thereon. In addition, the processormay enable execution of multiple programs, threads, and codes. Thethreads may be executed simultaneously to enhance the performance of theprocessor and to facilitate simultaneous operations of the application.By way of implementation, methods, program codes, program instructionsand the like described herein may be implemented in one or more thread.The thread may spawn other threads that may have assigned prioritiesassociated with them; the processor may execute these threads based onpriority or any other order based on instructions provided in theprogram code. The processor may include memory that stores methods,codes, instructions and programs as described herein and elsewhere. Theprocessor may access a storage medium through an interface that maystore methods, codes, and instructions as described herein andelsewhere. The storage medium associated with the processor for storingmethods, programs, codes, program instructions or other type ofinstructions capable of being executed by the computing or processingdevice may include but may not be limited to one or more of a CD-ROM,DVD, memory, hard disk, flash drive, RAM, ROM, cache and the like.

A processor may include one or more cores that may enhance speed andperformance of a multiprocessor. In embodiments, the process may be adual core processor, quad core processors, other chip-levelmultiprocessor and the like that combine two or more independent cores(called a die).

The methods and systems described herein may be deployed in part or inwhole through a machine that executes computer software on a server,client, firewall, gateway, hub, router, or other such computer and/ornetworking hardware. The software program may be associated with aserver that may include a file server, print server, domain server,internet server, intranet server and other variants such as secondaryserver, host server, distributed server and the like. The server mayinclude one or more of memories, processors, computer readable media,storage media, ports (physical and virtual), communication devices, andinterfaces capable of accessing other servers, clients, machines, anddevices through a wired or a wireless medium, and the like. The methods,programs or codes as described herein and elsewhere may be executed bythe server. In addition, other devices required for execution of methodsas described in this application may be considered as a part of theinfrastructure associated with the server.

The server may provide an interface to other devices including, withoutlimitation, clients, other servers, printers, database servers, printservers, file servers, communication servers, distributed servers andthe like. Additionally, this coupling and/or connection may facilitateremote execution of program across the network. The networking of someor all of these devices may facilitate parallel processing of a programor method at one or more location without deviating from the scope ofthe invention. In addition, any of the devices attached to the serverthrough an interface may include at least one storage medium capable ofstoring methods, programs, code and/or instructions. A centralrepository may provide program instructions to be executed on differentdevices. In this implementation, the remote repository may act as astorage medium for program code, instructions, and programs.

The software program may be associated with a client that may include afile client, print client, domain client, internet client, intranetclient and other variants such as secondary client, host client,distributed client and the like. The client may include one or more ofmemories, processors, computer readable media, storage media, ports(physical and virtual), communication devices, and interfaces capable ofaccessing other clients, servers, machines, and devices through a wiredor a wireless medium, and the like. The methods, programs or codes asdescribed herein and elsewhere may be executed by the client. Inaddition, other devices required for execution of methods as describedin this application may be considered as a part of the infrastructureassociated with the client.

The client may provide an interface to other devices including, withoutlimitation, servers, other clients, printers, database servers, printservers, file servers, communication servers, distributed servers andthe like. Additionally, this coupling and/or connection may facilitateremote execution of program across the network. The networking of someor all of these devices may facilitate parallel processing of a programor method at one or more location without deviating from the scope ofthe invention. In addition, any of the devices attached to the clientthrough an interface may include at least one storage medium capable ofstoring methods, programs, applications, code and/or instructions. Acentral repository may provide program instructions to be executed ondifferent devices. In this implementation, the remote repository may actas a storage medium for program code, instructions, and programs.

The methods and systems described herein may be deployed in part or inwhole through network infrastructures. The network infrastructure mayinclude elements such as computing devices, servers, routers, hubs,firewalls, clients, personal computers, communication devices, routingdevices and other active and passive devices, modules and/or componentsas known in the art. The computing and/or non-computing device(s)associated with the network infrastructure may include, apart from othercomponents, a storage medium such as flash memory, buffer, stack, RAM,ROM and the like. The processes, methods, program codes, instructionsdescribed herein and elsewhere may be executed by one or more of thenetwork infrastructural elements.

The methods, program codes, and instructions described herein andelsewhere may be implemented on a cellular network having multiplecells. The cellular network may either be frequency division multipleaccess (FDMA) network or code division multiple access (CDMA) network.The cellular network may include mobile devices, cell sites, basestations, repeaters, antennas, towers, and the like. The cell networkmay be a GSM, GPRS, 3G, EVDO, mesh, or other networks types.

The methods, programs codes, and instructions described herein andelsewhere may be implemented on or through mobile devices. The mobiledevices may include navigation devices, cell phones, mobile phones,mobile personal digital assistants, laptops, palmtops, netbooks, pagers,electronic books readers, music players and the like. These devices mayinclude, apart from other components, a storage medium such as a flashmemory, buffer, RAM, ROM and one or more computing devices. Thecomputing devices associated with mobile devices may be enabled toexecute program codes, methods, and instructions stored thereon.Alternatively, the mobile devices may be configured to executeinstructions in collaboration with other devices. The mobile devices maycommunicate with base stations interfaced with servers and configured toexecute program codes. The mobile devices may communicate on a peer topeer network, mesh network, or other communications network. The programcode may be stored on the storage medium associated with the server andexecuted by a computing device embedded within the server. The basestation may include a computing device and a storage medium. The storagedevice may store program codes and instructions executed by thecomputing devices associated with the base station.

The computer software, program codes, and/or instructions may be storedand/or accessed on machine readable media that may include: computercomponents, devices, and recording media that retain digital data usedfor computing for some interval of time; semiconductor storage known asrandom access memory (RAM); mass storage typically for more permanentstorage, such as optical discs, forms of magnetic storage like harddisks, tapes, drums, cards and other types; processor registers, cachememory, volatile memory, non-volatile memory; optical storage such asCD, DVD; removable media such as flash memory (e.g. USB sticks or keys),floppy disks, magnetic tape, paper tape, punch cards, standalone RAMdisks, Zip drives, removable mass storage, off-line, and the like; othercomputer memory such as dynamic memory, static memory, read/writestorage, mutable storage, read only, random access, sequential access,location addressable, file addressable, content addressable, networkattached storage, storage area network, bar codes, magnetic ink, and thelike.

The methods and systems described herein may transform physical and/oror intangible items from one state to another. The methods and systemsdescribed herein may also transform data representing physical and/orintangible items from one state to another.

The elements described and depicted herein, including in flow charts andblock diagrams throughout the figures, imply logical boundaries betweenthe elements. However, according to software or hardware engineeringpractices, the depicted elements and the functions thereof may beimplemented on machines through computer executable media having aprocessor capable of executing program instructions stored thereon as amonolithic software structure, as standalone software modules, or asmodules that employ external routines, code, services, and so forth, orany combination of these, and all such implementations may be within thescope of the present disclosure. Examples of such machines may include,but may not be limited to, personal digital assistants, laptops,personal computers, mobile phones, other handheld computing devices,medical equipment, wired or wireless communication devices, transducers,chips, calculators, satellites, tablet PCs, electronic books, gadgets,electronic devices, devices having artificial intelligence, computingdevices, networking equipments, servers, routers and the like.Furthermore, the elements depicted in the flow chart and block diagramsor any other logical component may be implemented on a machine capableof executing program instructions. Thus, while the foregoing drawingsand descriptions set forth functional aspects of the disclosed systems,no particular arrangement of software for implementing these functionalaspects should be inferred from these descriptions unless explicitlystated or otherwise clear from the context. Similarly, it will beappreciated that the various steps identified and described above may bevaried, and that the order of steps may be adapted to particularapplications of the techniques disclosed herein. All such variations andmodifications are intended to fall within the scope of this disclosure.As such, the depiction and/or description of an order for various stepsshould not be understood to require a particular order of execution forthose steps, unless required by a particular application, or explicitlystated or otherwise clear from the context.

The methods and/or processes described above, and steps thereof, may berealized in hardware, software or any combination of hardware andsoftware suitable for a particular application. The hardware may includea general purpose computer and/or dedicated computing device or specificcomputing device or particular aspect or component of a specificcomputing device. The processes may be realized in one or moremicroprocessors, microcontrollers, embedded microcontrollers,programmable digital signal processors or other programmable device,along with internal and/or external memory. The processes may also, orinstead, be embodied in an application specific integrated circuit, aprogrammable gate array, programmable array logic, or any other deviceor combination of devices that may be configured to process electronicsignals. It will further be appreciated that one or more of theprocesses may be realized as a computer executable code capable of beingexecuted on a machine readable medium.

The computer executable code may be created using a structuredprogramming language such as C, an object oriented programming languagesuch as C++, or any other high-level or low-level programming language(including assembly languages, hardware description languages, anddatabase programming languages and technologies) that may be stored,compiled or interpreted to run on one of the above devices, as well asheterogeneous combinations of processors, processor architectures, orcombinations of different hardware and software, or any other machinecapable of executing program instructions.

Thus, in one aspect, each method described above and combinationsthereof may be embodied in computer executable code that, when executingon one or more computing devices, performs the steps thereof. In anotheraspect, the methods may be embodied in systems that perform the stepsthereof, and may be distributed across devices in a number of ways, orall of the functionality may be integrated into a dedicated, standalonedevice or other hardware. In another aspect, the means for performingthe steps associated with the processes described above may include anyof the hardware and/or software described above. All such permutationsand combinations are intended to fall within the scope of the presentdisclosure.

While the invention has been disclosed in connection with the preferredembodiments shown and described in detail, various modifications andimprovements thereon will become readily apparent to those skilled inthe art. Accordingly, the spirit and scope of the present invention isnot to be limited by the foregoing examples, but is to be understood inthe broadest sense allowable by law.

All documents referenced herein are hereby incorporated by reference.

1. A computer program product embodied in a computer readable mediumthat, when executing on one or more computers, helps a user make adecision through the use of a computer facility by performing the stepsof: receiving an initial question from a user; providing the user with adialog consisting of questions from the computing facility and answersprovided by the user, wherein the computing facility utilizes at leastone of third party information, functions, and utilities; and providingthe decision to the user from the computing facility, wherein thedecision is based on the dialog and pertaining to the initial question.2. The computer program product of claim 1, wherein the computingfacility is a machine learning facility.
 3. The computer program productof claim 1, wherein the third party information consists of at least oneof product information from product manufacturers, product informationfrom web merchants, pricing information from other websites,availability information from other websites, pricing information frommerchants, availability information from merchants, a review, comments,and ratings.
 4. The computer program product of claim 1, wherein the atleast one of third party information, functions, and utilities includesan application programming interface (API).
 5. The computer programproduct of claim 4, wherein the API enables the collection of at leastone of cost information, product information, personal information, andtopical information.